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ABSTRACT 

Each year when the Educational Testing Services 
releases state average SAT (Scholastic Aptitude Test) scores of high 
school students, Indiana officials are asked why Indiana ranks so low 
and what's being done to improve it. Leaders reply that scores are 
lower because more students take the test than in states with higher 
scores. This report summarizes what can be learned from Indian's 
average SAT scores and what cautions are necessary to reach 
meaningful conclusions about the educational performance of Indiana 
students. Indiana high school seniors' average verbal-math SAT scores 
have historically ranked in the bottom one-third of the states. Two 
sources of bias that distort and lower Indiana students' average SAT 
scores are described — the varying rates of SAT participation among 
states, and differences among states' educational resources. It is 
argued that SAT scores offer a limited measurement of college-bound 
high school seniors' verbal and mathematics achievement, and do not 
accurately represent the state's educational performance. In 
addition, the SAT is used as a measure of student achievement, 
although its purpose is to assist colleges in making admissions 
decisions. Three different ways of reporting SAT scores are 
explained. It is concluded that the relative performance of Indiana's 
college-bound high school seniors improves in comparison to other 
states, once corrections are made. An education-performance report 
card is needed t d provide a comprehensive and informative view of 
Indiana education. (LMI) 
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Introduction 

An annual ritual occurs in Indiana each time 
the Educational Testing Service of Princeton, New 
Jersey, releases state average SAT scores of high 
scl ool students for verbal and mathematics 
achievement. Indiana school officials are asked why 
Indiana ranks so low among the states, and what is 
being done to improve education performance. 
School leaders reply that the SAT scores are lower 
because more students in Indiana take the test than 
in states with higher scores, and that programs 
designed to improve Indiana's test scores are in place. 
Some Hoosiers react by saying that schools need to 
improve, while others discount the Indiana SAT test 
results as another example of an inappropriate, if not 
misleading, measure of student achievement. What 
makes this annual ritual difficult to understand for 
many citizens interested in the performance of 
Indiana schools is that both parties are correct. The 
SAT does provide some useful information about the 
performance of Indiana's students, but without 
significant qualification and interpretation it also is 
a misleading indication of education performance in 
the Hoosier state. This report summarizes what one 
can !eam from state average SAT scores and what 
cautions are necessary to reach meaningful 
conclusions about the education performance of 
Indiana's students. 

What the SAT Tells Us About Indiana 

Indiana has historically ranked in the bottom 
one-third of the states in the average verbal-math 
SAT scores of its high school seniors. In 1992 
Indiana ranked 47th among the 50 states and the 
District of Columbia. 

Indiana's relative nosilion on the SAT among 
the states has remained about the same since 1972, 
the earliest year for which SAT scores are available, 
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Table 1 

Indiana Average SAT Scores, 1972-1992 



Year 


A vprnop 

Verbal- 
Math 
Score 


State 
Rank 


Cream 

of fhp 

Crop 
Adjusted 
Rank 


Resources 
and 
Cream 

Crop 
Adjusted 
Rank 


1992 


868 


47 






1991 


865 


47 






1990 


867 


47 






-1989 


871 


47 


36 


26 


"1988 


870 


47 


38 


30 


^ 1987 


874 


47 


38 


29,5 


*!986 


874 


47 


36 


29.5 


*1985 


875 


47 


36 


24.5 


* 1984 


864 


47 


36 


28 


1983 


865 


46 






* 1982 


860 


46 


35 


28 


1981 


S57 


46 






1980 


358 


47 






1979 


867 


46 






1978 


870 


46 






1977 


870 


47 






1976 


875 


47 






1975 


881 


46 






1974 


892 


46 






1073 


899 


44 






1972 


906 


45 






*Note: Adjustments made for years where ACT data are 

available. 



even though national SAT average scores have 
declined during the last 20 years ( See Table 1). 
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Does this rather low relative performance on 
an achievement test mean that Indiana's school-aged 
population has poor education performance? The 
answer is, "Not necessarily/' The current practice of 
reporting these average test scores introduces two 
known sources of bias that distort and lower the 
average test scores of Hoosier high school students 
when compared to other states. These two biases are 
the result of varying SAT participation rates and the 
different resources invested in state education 
systems. 

The Effects of SAT Participation 

In a 1 985 Harvard Education Review research 
article Powell and Steelman (1984) documented that 
the average ",tate SAT score decreases as a larger 
percentage of students take the SAT. In other words, 
when smaller percentages of students, usually higher 
achievers, take the test, the average test score in the 
state will be higher. The pattern observed by Powell 
and Steelman is illustrated in Figure 1, which shows 
the relationship between the percent of eligible 
students taking the test and the average state SAT 
score for 1992. 

This pattern may be called the "cream of the 
crop" phenomena, since the evidence suggests that 
the more academically talented and high achieving 
high school students are likely to take the SAT for 
college admission first, and these test takers are not 
typical of the student body of eligible students. The 
other national verbal-math achievement test used for 
college admission decisions (besides the SAT) is the 
ACT. The SAT is the test of choice in 22 states 
including Indiana, and the ACT in 28 states. 1 Since 
more than half of the states (28) have low 
percentages of eligible students taking the SAT 
(students in these states usually take the ACT), the 
average SAT scores from these states will be 
artificially inflated, while in the 22 SAT states, the 
average test scores will be lower because a more 
representative cross-section of eligible students take 
the test. The states adjacent to Indiana — Michigan, 
Illinois, Kentucky, and Ohio — historically have had 
SAT scores higher than Indiana, because students in 
these states are more likely to take the ACT for 
college admission and thus only a small percentage 
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Figure 1 

Average SAT Scores by SAT Participation Rate 
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take the SAT. In summary, the "cream of the crop" 
tends to take the SAT in 28 ACT states, whereas a 
more representative proportion of students take the 
SAT in 22 states, including Indiana. 

The SAT States Are Better Positioned 
to Support Education 

The 22 SAT states differ in another 
significant way from the 28 ACT states. The SAT 
states invest more resources into their education 
systems and have more favorable social-economic 
conditions for education than the 28 ACT states. 
The SAT states tend to have less poverty, higher 
personal incomes, higher education spending per 
pupil, and smaller pupil/teacher ratios than the ACT 
states. Since money, class size, lack of poverty and 
high personal income are each related to improved 
performance of the SAT, these different resource and 
socio-economic conditions in the states must be 
considered in order to compare SAT averages 
among the states. 



In a recent Policy Studies Journal article 
(Lehnen, 1992) the author found that these resource 
and funding variations accounted for differences 
between the SAT and ACT states from 23 to 38 SAT 
points. This is a sizeable difference in average SAT 
scores if one recognizes that Indiana's average SAT 
score usually fluctuates less than 5 points from year 
to year. If one compares ACT and SAT states 
directly, without adjusting for these differences, 
Indiana's average test scores are 13 to 30 test score 
points lower than the ACT states. 

What Can We Learn abaut 
Education Performance from SAT Scores 

To compare average SAT scores among the 
states, one must understand which students take the 
SAT and what the test actually measures about 
student performance. Not all Hoosier students take 
the SAT. In fact, the SAT reports average test 
scores for high school seniors who may wish to 
attend college. The SAT does not measure the 
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performance of students in the lower grades (K 
through 11), nor for students unlikely to enroll in 
higher education. The education performance of 
students who do not attend college, who enter the 
military or the workforce, or who drop out of high 
school is not measured by SAT scores. The group 
measured by average SAT scores then is "college- 
bound high school seniors." When a sufficiently large 
percentage of this group, about 30% or more of 
eligible students, takes the SAT, one can obtain a 
reasonable estimate of their education performance. 
But what kind of education performance is measured 
by the SAT? 

The SAT is an achievement test that primarily 
measures verbal and math achievement, including 
reading comprehension, vocabulary, math 
computation, and math problem solving. The basic 
test does not directly measure learning in social 
studies or in the arts and humanitLs areas, nor does 
it measure higher-order reasoning, group problem- 
solving, workforce readiness skills, citizenship, and 
other knowledge taught in grades K-12. Most 
importantly, the SAT does not necessarily reflect the 
unique and varied curriculum in each of the states — 
that is, what the students are actually taught even in 
the areas of verbal and math achievement. The reason 
is that the SAT was never designed to be a measure 
of education performance. Rather the sole purpose of 
the SAT is as a tool used by some colleges and 
universities to assist them in making admission 
decisions. Although the SAT is used as a measure 
of student achievement, its purpose has always been 
as an aid to college admissions decisions. The SAT 
then is a limited measure of education achievement 
in verbal and math areas of college-bound high 
>chool students. 

A Revised View of Indiana's SAT Performance 

The ranking of Indiana's college-bound high 
school seniors at about 47th among the states is 
artificially low in light of the "cream of the crop" 
phenomena and differences in education input 
resources available to the states. But how favorably 
do Indiana's college-bound high school seniors 
compare once these differences are accounted for? 
The following table summarizes three different ways 



of reporting state average SAT scores. 



Table 2 

Methods of Reporting SAT Scores 
by Type of Adjustment 



Methods ot 
Reporting 


"Cream of the 
Crop" Problem 


Input Resource 
Differences 


Average SAT 
lest score 
reported by the 
Educational 
Testing Service 


No correction 
made 


No correction 
made 


Adjusted for 
self-selection 
only 


correction made 


No correction 
made 


Adjusted for 
resource inputs 
and self-selection 


correction made 


correction made 



Indiana's ranks based on these three methods 
of reporting SAT achievement test results are 
summarized in Table 1. The main point of Table 1 
is that the relative performance of Indiana's college- 
bound high school seniors improves in comparison 
to other states once corrections are made, since 
Indiana moves from a rank of 47th among the states 
to a rank varying between 24th and 38th, depending 
on the choice of corrections and the year selected. 
The reason that Indiana's relative position improves 
is that the corrections adjust the ranks of ACT states 
such as Illinois, Michigan, Kentucky, and Ohio, 
downwards. The "good news 0 is that Indiana 
historically has not ranked near last place on SAT 
results, but the "bad news 0 is that Indiana still ranks 
in the bottom-half of the states. 

Some Implications for 
Indiana's Education Policy Makers 

The 1989 Indiana General Assembly 
mandated aiat Indiana school districts report average 
SAT scores of students in local newspapers each 
year. Each August Hoosiers will find reports 
published in local newspapers on their district's SAT 
scores. This information is now beginning to be 
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used by education stakeholders in Indiana 
communities to make the case for and against the 
need to improve education. It is fair to say that the 
problems of reporting and interpreting SAT scores at 
the school district level are as difficult, if not more 
so, than at the state level. One must ask whether 
reporting SAT scores at the district level provides 
sufficient and accurate information about student 
achievement in the district. 

The issue is not so much whether to report 
SAT test score averages but whether the public 
interest is served by reporting district SAT scores 
without providing any interpretation and analysis. 
Education stakeholders and other citizens have 
common interests in developing facts and figures on 
education performance that inform them about 
Indiana education. The Indiana General Assembly 
should consider what kinds of district-level school 
performance measures supplemented with analysis 
and interpretation are needed to inform citizens and 
enhance public understanding about local education 
issues. Simply publishing district SAT scores in local 
newspapers does not inform the public about the 
performance of its schools. 

The best that can be said for SAT scores is 
that they provide, with some corrections for known 
distortions, limited information about the verbal- 
math achievement of a small number of students 
(college-bound high school seniors). What Indiana 
lacks is an Education Performance Report Card that 
provides a comprehensive and informative view 
about education in Indiana. This report card should 
identify the characteristics of Indiana's schools that 
Hoosiers wish to know about — say, achievement test 
scores and graduation rates, revenues and 
expenditures, the number of at-risk and special needs 
children, the training and experience of the faculty 
and staff, and the social conditions of the area served. 
The information about these characteristics must be 
assembled, analyzed, and reported for each school 
district in an impartial way. When this is 
accomplished, the annual discussion about whether 
Indiana really is 47th on the SAT will cease to have 
relevance. 



Endnotes 

1. This classification of states is the approach used by 
the U.S. Department of Education "wall chart" 
reports (USDOE, 1984-1989). The state of 
Washington does not have large numbers of students 
taking either the SAT or ACT. 

2. These corrections were developed by the author 
based on the average state SAT test scores reported 
in the u wall chart." Average ACT score by state 
were released for the years 1972, 1982, and 1984 
through 1989 only and are not available for the 
remaining years. See Lehnen (1992). 
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